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ABSTRACT 

An analysis of the distribution of Killam Research Fellowships awarded by 
Canada Council over a seventeen-year period reveals a pattern which is examined 
in relation to the distribution of applications from universities. The existing 
situation is discussed with reference to previous studies of peer review systems and 
to implications for Canadian research. Encouragement of increased participation 
across Canada is suggested as one possible concern for Canada Council policy. 


RESUME 

Les resultats d’ une analyse de la repartition des bourses de chercheur-boursier de 
la fondation Killam, allouees par le Conseil des Arts du Canadasur une periode de 
dix-sept annees, et la repartition des demandes de bourses provenant de 
differentes universites ont fait Tobjet d' une etude comparative. L’on discute de la 
situation actuellement par rapport a des etudes prealables sur les mecanismes 
d’evaluation par les pairs et leurs repercussions sur la recherche au Canada. On 
suggere que la politique du Conseil des Arts encourage, entre autres, une 
participation active et grandissante dans tout le Canada. 


Questions about priorities in the allocation of research funds, and the 
advantages of large university centres as against dispersion at many locations were 
discussed by Weinberg (1963, 1964, 1974) and Singer (1982). We wish to raise 
similar questions about the allocation of research funds in Canada Council’s 
Killam Research Fellowships programme for university faculty (see “Killam 
Programme”, Ottawa, Canada Council, 1986). More particularly, we ask the 
following questions: 

1. What are the differentials in the research participation and success of Canadian 
universities? 


*John K. Chadwick-Jones, Saint Mary’s University, Halifax, A. Carceller, Mount Saint Vincent 
University, Halifax and Douglas Vaisey, Saint Mary’s University, Halifax. 






28 John K. Chadwick-Jones, A. Carceller and Douglas Vaisey 


2. Are the stated objectives of the funding agency being attained most effectively? 

If there is a moral order of research fund allocation, then it seems clear that this 
should be based on a well-defined reviewing process for individual applications 
leading to selection of the most meritorious. This selection process should involve 
assessment by experts who consider each applicant’s research proposal together 
with referees’ comments. The procedure should follow systematic rules so that 
decisions are taken as rationally as possible. 

There are probably difficult choices to be made in this (Killam) competition 
with a large number of applicants and a relatively small number of awards. The 
funding agency must have some notions of fair process as guides for its judging 
committees as they proceed to the point of allocating awards to individuals by vote 
or consensus. Probably the process itself is complex, because there may be a 
number of dimensions on which each proposal is to be judged (e.g. applicant’s 
capacities, realistic plan, appropriate budget, sufficient detail) and choices have 
to be made between disciplines and sub-disciplines, or between pure and applied 
orientations. At the moment we do not have information from Canada Council on 
how these choices are made. 

Any system of funding usually and probably inevitably will produce unfair 
negatives for some applicants - if only because no system can operate perfectly 
according to rational-decision rules (Bauer, 1968). However, one aim of 
examining how a given system of funding actually operates, is to discover whether 
the likelihood of inequitable losing is influenced by chance factors or by some 
biassing factors which might disadvantage some applicants. In the Killam 
fellowships programme, certain universities provide many of the successful 
applicants, but this may indicate only that these applicants are of better quality. On 
the one hand, if the limitation of awards to a few universities were so extreme as to 
suggest a perfect correlation of talent with university affiliation, then the existing 
policies might be questioned, legitimately. On the other hand, a university 
probably needs some concentration of resources and of faculty members (a notion 
akin to “critical mass”) before it can generate applicants to win Killam 
fellowships. In this kind of situation, the distribution of awards, even if it seems 
skewed in favour of some universities, may be the result not so much of decisions 
taken by the selection committees as an unavoidable outcome of the current size 
and organization of Canadian universities. 

In the administration of any awards system, there must be accountability. This is 
a main issue of our analysis and accountability may require re-appraisals with, 
eventually, modifications of existing procedures. However, no criticism of the 
selection procedures that are currently in use is implied in the analysis that follows. 


The Killam Programme 

The Killam Research Fellowships programme of the Canada Council is, according 
to its brochure for applicants, “intended to support scholars of exceptional ability 
engaged in research projects of broad significance and wide-spread interest” 
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TABLE 1 

KILLAM FELLOWSHIP (FIRST YEAR) AWARDS 1968-1984 
TOTAL AWARDS GRANTED (188) 


Universities 

No. of fellowship! 
received 

Toronto 

55 

British Columbia 

21 

McGill 

16 

Queen's 

13 

York 

13 

Montreal 

11 

Laval 

9 

McMaster 

7 

Ottawa 

5 

Manitoba 

4 

Simon Fraser 

4 

Western Ontario 

4 

Private Scholars 

2 

14 Other Universities 
Receiving 1-3 Awards 

24 

TOTAL 

188 


through replacement of salary, enabling periods of leave to be taken, usually of 
one or two years. The Killam awards were made possible by a bequest of Mrs. 
Dorothy J. Killam. Between 12-20 fellowships are given per year, each re¬ 
newable for a second year: the total annual expenditure is more than two million 
dollars currently. In Dorothy Killam’s own words: “my purpose in establishing the 
Killam trust is to help in the building of Canada’s future by encouraging advanced 
studies. Thereby, I hope to increase the scientific and scholastic attainment of 
Canadians, to develop and to expand the work of Canadian universities...” 
Nothing is said in the brochure about the distribution of awards across universities 
and provinces of Canada. 

In fact, as will be shown, some universities receive a high proportion of the 
awards. The pattern of the Killam awards shows, in effect, a 3-tier structure of the 
Canadian universities: the first tier consisting of a few universities with a high 
proportion of successful applications (possibly those with the highest concentra¬ 
tion of talent). The University of Toronto has received about one-third of all 
awards - 29 percent over the total period of 17 years (Table I) and 30 percent of 
awards made in the last 5 years of the analysis (Table II). The University of British 
Columbia and McGill University (Quebec) each received around 10 percent over 
the total period and between 13 and 17 percent for the last 5 years. Queen’s and 
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TABLE II 

KILLAM FELLOWSHIP (FIRST YEAR) AWARDS 1980-1984 
TOTAL AWARDS GRANDED (57) 

N o. of fellowships 


Universities received 

Toronto 17 

British Columbia 9 

McGill 7 

Queen's 5 

Ottawa 4 

York 3 

Manitoba 3 

Western Ontario 2 

Simon Fraser 2 

Alberta 1 

McMaster 1 

Waterloo 1 

Concordia 1 

Sherbrooke 

TOTAL 57 


York Universities in Ontario, also did relatively well, each with 7 percent of the 
total awards. Below these are a group of universities with modest success (14 of 
them with only 1-3 awards over the entire period (Table I). The third tier consists 
of universities with no awards. 

Altogether, the University of Toronto obtained 55 of the total 188 awards. If the 
University of Toronto share is added to that of the University of British Columbia, 
McGill and Queen’s, then the summation comprises well over half of all awards 
(Tables I, II). Analysis of total funding confirms the general picture. The 
University of Toronto has received 35, 35, 32, 31 and 24 percent of the total funds 
over the five years. During the same period the nearest runner-up was the 
University of British Columbia with 15 percent and 14 percent of the total funds in 
1981 and 1982 (Table III). 

It might be uncharitable to assume that the universities whose faculty numbers 
have not achieved Killam fellowships have little or no research output. In a Canada 
Council report of 1977 the authors cite a survey of small universities where “most 
faculty members interviewed felt that research forms an essential part of any 
university department” but the survey also noted that “research" was interpreted in 
these institutions in a broader sense to the definition used in “the most prestigious 
academic circles”. This broader definition included “applied research, literature 
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TABLE III 

FINANCIAL DISTRIBUTION OF AWARDS: LARGEST ALLOCATIONS 1980-84 
PERCENTAGES OF TOTAL FUNDS ALLOCATED IN KILLAM PROGRAMME 


1980 

1981 

1982 

1983 

1984 


University of 

Toronto 

35 

35 

32 

31 

24 

University of 

British Columbia 

4 

15 

14 

11 

12 

Queen's University 

11 

11 

7 

6 

15 


reviews, course development and small-scale experimentation”. Also, there is no 
doubt that many universities produce few applicants for the Killam programme. 
Information on applications has been made available to us (back to 1977) by the 
Canada Council (Table IV). 

Altogether, over the period 1980-84, applications were received from faculty 
members in 36 universities. The University of Toronto provided, consistently, 
thirty or more applications per year, submitting 22 percent of the total (172 out of 
787) during the five years, 1980-84. The relatively low applications rate of a large 
number of “also ran” Canadian universities, even the larger ones, in this 
competition contrasts with the high participation by the University of Toronto, 
well ahead of the University of British Columbia (63 applicants) and McGill 
University (44 applicants). One interesting feature of the period 1980-84 is the 
absence of awards to the francophone universities of Quebec although these 
universities did gain awards during the first decade of the analysis (Table I). 
Possibly, able individuals in this province may have turned to increased research 
funding from provincial sources for Francophone scholars in Quebec but this must 
remain conjectural given the limits of our current analysis. 

The first-time fellowships are in most cases renewed for a further year. 
However, some have been made for more than two years. For example, one 
individual at McGill University had his fellowship renewed for four consecutive 
years 1970-74 and received a new fellowship in 1976, renewed in 1977. Another 
McGill University scholar’s fellowship of 1973-74 was renewed for three further 
years from 1974 to 1977. A Queen’s University recipient received fellowships 
over the five years (1968-72) and a University of Toronto recipient’s fellowship 
lasted for four years (1975-79). These multiple awards serve to emphasize the 
allocations of available funds to a few universities even more than has appeared 
from the distribution of the initial awards. 

The selection committee of 12-13 members is the principal instrument for 
awards: most are academics, with 1-3 ex-officio members (one is secretary of the 
Killam programme). Since the beginning of the competition in 1968, 27 
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TABLE IV 

APPLICATIONS RECEIVED: KILLAM RESEARCH FELLOWSHIPS 


Year 

No. of universities 
from which applications 
were received 

Total applicants 

1980 

27 

138 

1981 

27 

126 

1982 

30 

139 

1983 

30 

167 

1984 

28 

217 


universities have been represented on this committee, a wide spread of 
representation. The University of Toronto is the only one that has been represented 
on the committee in each of the 17 years (1968-84); for 7 years it had 2 
representatives and in 3 years it had 3 representatives (1976-77, 1981-82, 
1982-83). In the last decade McGill University has had representatives during 
each year, with 2 representatives during 6 of those years. 

That members of the selection committee, reviewers and applicants may be 
personally known to each other does not constitute evidence of bias. On the 
contrary, such personal contacts are often seen as positive in assisting the decision 
process and, in any case, are observed in a variety of research funding contexts 
(Hall, 1986). There is still the question of whether the existing distribution of 
awards achieves the specific aim of the Killam trust “to develop and expand the 
work of Canadian universities”. Or is it the case that the resources of the Killam 
programme, available by open competition to the wider academic community 
across Canada, actually support the research activities of a minority of universities 
rather than encouraging the general development of research skills in Canada? 

In the Killam fellowship programme, the Prairies and Atlantic provinces receive 
few awards. For example, from 1980 to 1984 four Prairie universities (the 
Universities of Alberta, Calgary, Manitoba, Saskatchewan) made 77 applications 
and received four initial awards. Eight of the Atlantic universities (Dalhousie, 
Memorial University of Newfoundland, Mount Allison, St. Francis Xavier, Saint 
Mary’s, Mount Saint Vincent, Universite de Moncton, University of New 
Brunswick) submitted a total of 37 applications; they received no awards. Queen’s 
University in the same period submitted 44 applications and received 5 awards. 

Further light might be thrown on this question in the future, if Canada Council 
were to consider carrying out an enquiry into the quality of applications from the 
Atlantic provinces. We can examine the participation of these eight Atlantic 
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TABLE V 

KILLAM FELLOWSHIP (FIRST YEAR) AWARDS: 

UNIVERSITY COMPARISONS 
1980-1984 

Universities 

Toronto Queen's Prairies (4) Atlantic (8) 


No of Applications 172 44 77 37 

No. of Successful 

Applications 17 5 4 0 

Percentage of 
Successful 

Applications 9.8 11.4 5.2 0 


universities. If we take the University of Toronto as our model (using data from 
Table V) then let us suppose that each university in the Atlantic provinces should 
provide annual applications at an equivalent rate (assuming, that is, their faculty 
members are research-oriented to a similar extent). Based on the University of 
Toronto participation (Table V), this yields us a formula for the expected 


participation of: 

(No. of applicants 
172/5 over 5 years) 

No. of full-time 

Expected no. 

2,373 (Total faculty members 

faculty 

of applications 

U. of Toronto)* 




By this formula, a small university with, say, 275 faculty would need to supply 4 
applications per year to have an equal rate of participation. For the Atlantic 
universities in the five years 1980-84, 37 applications have been submitted from 
eight universities with a total faculty of approximately 3,107 members. Applying 
the formula, the resulting figure of expected applications would be: 


172/5 

-x 

2,373 


3,iO 7 

—-—= 45 per year 


The actual participation rate is, thus, very much lower than the expected level. 
In fact the total applications made in the five years are less than the suggested rate, 
using the above formula, for only one year’s applications. 


“Source of figures for 1983-84: Statistics Canada. Salaries and Salary Scales of Full-Time Teaching 
Staff at Canadian Universities (CS81-258) Ottawa: Statistics Canada. Figures include medical and 
dental schools. 
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Discussion 

The success of members of major universities in the Killam competition merits 
comparison with similar tendencies documented in discussions of peer review 
systems for scientists in the United States. The literature on peer review has 
concentrated on the sciences; we do not have equivalent information on the 
humanities and social sciences, so the relevance of this literature for the Killam 
awards allocations is only partial. Cole and Cole (1967, 1968) considered the 
hypothetical situation of an individual who produces high quality work but is a 
member of a low prestige department; they identified university affiliation as an 
important influence on success. The Coles found, in their study of 120 physicists, 
that there was a high correlation between scientific recognition and prestige rank of 
department. They suggested also that published work by members of major (as 
contrasted with minor) universities gains most attention. However, Mitroff and 
Chubin (1972) discussed two studies of peer review in the National Science 
Foundation and in these studies the results were rather less decidedly in favour of 
the effect of institutional prestige. More recently, Belanger and Lacroix (1986) 
have discussed the likelihood of institutional status effects on bias in the wider 
context of the federal grants agencies of Canada and they have discounted 
institutional influence, pointing to the importance of individual productivity: “the 
correlation between publication productivity and grant recipientship is very high 
and credible” (p. 27). Thus, there is no evidence of institutional bias in the 
incomparably larger scene of federal funding, where the individual productivity of 
applicants appears to be the decisive influence. 

On the other hand, in the United States, Crane (1965) had marshalled evidence 
showing that scientists at major universities were more likely to be productive and 
to receive recognition than those located at minor universities. The more 
prestigious institutions were considered to provide an encouraging ambiance for 
research, including advantages gained from the presence of successful senior 
academics, who could provide “an adequate model for the optimum way to 
proceed”. In addition Crane noted that the effect of location “may actually be due 
to contacts it facilitates between junior men and eminent scientists who play 
important roles in allocating scientific rewards”. This kind of contact was thought 
also to influence the selection of good research topics by applicants at major 
universities and to increase a researcher’s “visibility” in his/her profession. 

According to a Canada Council report of 1977, the faculty members of smaller 
universities suffer isolation because of lack of departmental colleagues in the same 
field; because of heavy teaching loads and lack of opportunities to make academic 
contacts at other universities. Research productivity, according to Crane (1965), 
“did not make the scientist as visible to his colleagues as did a position at a major 
university”. Scientific recognition was the subject of several conjectures by Crane, 
for example: 1. recognition may be relative to achievement, 2. location at a major 
university creates a “halo” effect so that a scientist’s work looks better than it 
actually is, 3. contact with eminent colleagues enhances visibility and affects the 
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chances for recognition. Crane’s discussion concluded with speculations about 
possible future improvements in standards outside the group of elite institutions 
and made the prediction of an expansion of research activity beyond the major 
institutions. 

The study by Zuckerman and Merton (1971) of peer review systems for 
selection of articles for scientific journals also has some interesting comments 
about successful performance. Scientists at major universities may be more 
successful, they suggest, “because their critical associates (university colleagues) 
and they themselves have demanding internal standards, then the manuscripts they 
decide to submit for publication are apt to be rigorously pre-selected, with 
consequently high rates of acceptance by referees applying similar ... criteria”. It 
could also be, as was mentioned by Crane (1965), that scientists who have the 
advantage of the criticism and help of such colleagues will also tend to select more 
attractive research problems. 

To summarize, we have discussed the awards distribution of the Killam research 
fellowships and have shown evidence of a pattern which may, in fact, be the best 
outcome under the existing condition of the competition. The faculty members of 
some universities are continually successful in the Killam fellowships competi¬ 
tion. It is also the case for other universities that, overlong periods, relatively few 
applications are submitted. Could the stated objective of the Killam research 
fellowships programme “to develop and expand the work of Canadian universi¬ 
ties” be attained more effectively by a wider distribution of awards? There is also a 
second question: could the participation of Canadian universities be increased? 
One situation, perhaps comparable, that comes to mind is the concentration of 
French scholarship and research at the University of the Sorbonne in Paris. Singer 
(1982) has discussed the options of funding institutions at the centre, as against 
spreading the research allocations across a number of locations. He ventures the 
opinion, in the case of French universities, that the encouragement of academic 
talents at provincial locations would increase research output on a greater scale 
“than concentration in one location”. In our case, even if we were to regard this 
kind of opinion favourably, there is the problem of how to encourage, in the 
Killam competition, higher rates of participation across Canada. It might be a 
problem worthy of consideration in Canada Council policy. 
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